Skip to content

fix: wait for HTTP server drain during graceful shutdown#1745

Open
AmanGIT07 wants to merge 2 commits into
mainfrom
fix/connect-shutdown-drain
Open

fix: wait for HTTP server drain during graceful shutdown#1745
AmanGIT07 wants to merge 2 commits into
mainfrom
fix/connect-shutdown-drain

Conversation

@AmanGIT07

Copy link
Copy Markdown
Contributor

Summary

Graceful shutdown now finishes draining in-flight requests before the
process tears down its dependencies.

Changes

  • ServeConnect tracks its shutdown goroutines with a sync.WaitGroup
    and waits for them after ListenAndServe returns.
  • A failed server.Shutdown returns early instead of also logging the
    "shutdown complete" message.
  • The metrics server shutdown gets the same grace period as the connect
    server, and its error is logged instead of dropped.

Technical Details

http.Server.Shutdown closes the listener first and drains active
requests afterwards, so ListenAndServe returns before the drain
finishes. ServeConnect now waits for the shutdown goroutines, so it
only returns once requests are done.

Test Plan

  • Added TestServeConnectReturnsAfterShutdownOnContextCancel, runs
    the real server with and without the metrics listener
  • go test -race ./pkg/server/... passes
  • golangci-lint run pkg/server/ reports 0 issues

🤖 Generated with Claude Code

ServeConnect returned as soon as the listener closed, before in-flight
requests finished draining, so callers tore down the database while
requests were still running. Track the shutdown goroutines with a
WaitGroup and wait for them before returning. Bound the metrics server
shutdown with the same grace period and log its error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
frontier Ready Ready Preview, Comment Jul 9, 2026 12:15pm

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f2d37794-62c6-4f5a-a735-71a5e95c83f9

📥 Commits

Reviewing files that changed from the base of the PR and between 05a9de9 and 2075906.

📒 Files selected for processing (1)
  • pkg/server/server.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/server/server.go

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Improved graceful shutdown for the main service and the optional metrics endpoint when the app is closed.
    • Reduced the chance of requests or connections being left incomplete during shutdown.
    • Added automated coverage to verify the service exits cleanly after context cancellation in both core-only and metrics-enabled modes.

Walkthrough

ServeConnect now coordinates graceful shutdown for the connect and optional metrics HTTP servers with a shared WaitGroup, uses timed shutdown contexts, logs shutdown outcomes, and waits for shutdown work to finish before returning. A new test covers context-cancelled shutdown for both server configurations.

Changes

ServeConnect Shutdown Coordination

Layer / File(s) Summary
WaitGroup-based shutdown implementation
pkg/server/server.go
Adds a shutdownWG sync.WaitGroup, wraps connect and metrics server shutdowns with timed context and error logging, and waits for shutdown completion before ServeConnect returns.
Shutdown behavior test
pkg/server/server_test.go
Adds freePort and waitForHTTP helpers and a test verifying ServeConnect exits without error within 15 seconds after context cancellation, for both connect-only and connect+metrics scenarios.

Estimated code review effort: 2 (Simple) | ~15 minutes

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
pkg/server/server.go (1)

268-276: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Early return on ListenAndServe failure skips shutdownWG.Wait.

If ListenAndServe returns a non-ErrServerClosed error (e.g., port bind failure), ServeConnect returns at line 269 without waiting for the shutdown goroutine, which is still blocked on <-ctx.Done(). The goroutine will eventually unblock when the caller cancels ctx, but it outlives the function call. Consider returning after shutdownWG.Wait() on this path as well, or documenting that the error path intentionally skips the wait.

🔧 Proposed fix to wait on all return paths
 	// Start server
 	if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
+		// Ensure shutdown goroutines complete even on the error path.
+		shutdownWG.Wait()
 		return fmt.Errorf("connect server failed: %w", err)
 	}

Note: the caller must cancel ctx for the shutdown goroutines to unblock; if they don't, Wait will block indefinitely. Alternatively, keep the early return but document the intentional skip.

🧹 Nitpick comments (1)
pkg/server/server_test.go (1)

43-87: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider testing actual in-flight request drain behavior.

The test verifies ServeConnect returns after context cancellation, but doesn't verify the core PR claim: that in-flight requests finish draining before ServeConnect returns. A long-running handler that blocks until a signal would prove the drain actually waits. Without this, a regression that removes shutdownWG.Wait() would still pass this test (since Shutdown closes the listener quickly).

💡 Suggested drain-verification test case
func TestServeConnectDrainsInflightRequests(t *testing.T) {
	logger := slog.New(slog.NewTextHandler(io.Discard, nil))

	var cfg Config
	cfg.Connect.Port = freePort(t)

	// Register a slow handler via a custom mux that ServeConnect can use.
	// Since ServeConnect builds its own mux, you'd need to either:
	// 1. Add a test-only hook to inject handlers, or
	// 2. Use the /ping endpoint with a deliberate delay via a wrapper.
	//
	// Minimal approach: verify that after cancel, an in-flight /ping
	// request still completes successfully.

	// This would require either a test hook in ServeConnect or
	// a separate test that directly exercises server.Shutdown + WaitGroup.
}

This may require a small test hook in ServeConnect to inject a slow handler; if that's too invasive, consider a unit test for the shutdown coordination logic in isolation.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f1a4f9c2-0e14-4c56-b9e7-c5b10d0ebd5e

📥 Commits

Reviewing files that changed from the base of the PR and between 8ad9342 and 05a9de9.

📒 Files selected for processing (2)
  • pkg/server/server.go
  • pkg/server/server_test.go

Comment thread pkg/server/server.go Outdated
@coveralls

coveralls commented Jul 9, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 29017513952

Coverage increased (+0.3%) to 45.205%

Details

  • Coverage increased (+0.3%) from the base build.
  • Patch coverage: 5 uncovered changes across 1 file (14 of 19 lines covered, 73.68%).
  • 1 coverage regression across 1 file.

Uncovered Changes

File Changed Covered %
pkg/server/server.go 19 14 73.68%

Coverage Regressions

1 previously-covered line in 1 file lost coverage.

File Lines Losing Coverage Coverage
pkg/server/server.go 1 65.67%

Coverage Stats

Coverage Status
Relevant Lines: 37609
Covered Lines: 17001
Line Coverage: 45.2%
Coverage Strength: 12.68 hits per line

💛 - Coveralls

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants